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Abstract 



. In this article we focus on the problem of channel decoding in presence of a-priori information. In particular, 

' assuming that the a-priori information reliability is not perfectly estimated at the receiver, we derive a novel analytical 

framework for evaluating the decoder's performance. It is derived the important result that a "good code", i.e., a 
Q ' code which allows to fully exploit the potential benefit of a-priori information, must associate information sequences 

. with high Hamming weights to codewords with low Hamming weights. Basing on the proposed analysis, we analyze 

the performance of convolutional codes, random codes, and turbo codes. Moreover, we consider the transmission 
of correlated binary sources from independent nodes, a problem which has several practical applications, e.g. in 
the case of sensor networks. In this context, we propose a very simple joint source-channel turbo decoding scheme 
where each decoder works by exploiting a-priori information given by the other decoder. In the case of block fading 
channels, it is shown that the inherent correlation between information signals provide a form of non-cooperative 
diversity, thus allowing joint source-channel decoding to outperform separation-based schemes. 



I. Introduction 



^ ■ In most digital applications source and channel coding are treated as separate schemes, and the common approach 
^ of channel coding is to consider source encoded streams as statistically independent streams. However, in several 
situations it is not possible, or not convenient, to let source coding eliminating all intrinsic data redundancy. In this 
cases, the decoder can exploit such a residual (or total) redundancy in its effort of combating noise by performing 
joint source-channel decoding (JSCD). However, one of the main problem which arises in JSCD is represented 
by implementation complexity of the decoder, which in general increases to take into account the memory of the 
information source. As an example, for a first-order Markov source which is protected by convolutional codes, 
the optimum ISCD scheme is the maximum a posteriori (MAP) sequence decoder based on a super-trellis. The 
. ^ _ number of the super-trellis states is the product of the number of states of the convolutional trellis and the Markov 
^ ■ trellis. Some methods have been proposed to reduce the number of trellis states, which result in suboptimum MAP 
^ decoders based on symbol or bit-level lUl, ||2l, fSl, S, HI, lH. Suboptimal codes aim at presenting redundancy 
" " ' of information sources as a-priori information (API) at the input of channel decoder/demodulator, so that iterative 
schemes can be easily derived where at each iteration API can be easily enclosed in the decoder without substantially 
increasing the receiver complexity. In particular, when API is presented at bit-level, the use of channel decoding 
schemes can be easily extended to all MAP-based decoding schemes, e.g., turbo decoders and LDPC decoders [T], 

m. 

Another field where ISCD is gaining its momentum is the transmission of detected signals observed at different 
nodes in Wireless Sensor Networks (WSNs) Q. In the case of a single collector node (the access point), the study 
of efficient transmission mechanisms is often referred to as reach-back channel problem lITOl . lITTI . |[T2l . In an 
attempt to exploit the intrinsic correlation among data, many works have recently focussed on the design of source 
coding schemes that approach the Slepian-Wolf fundamental Umit on the achievable compression rates llT3l . |[T4l . 
lITSl . |[T6l . thus applying the separation principle. However, the design of good practical source codes for correlated 
sources is still an open problem. Besides, separation between source and channel coding may lead to catastrophic 
error propagation. Eventually, the traditional code design requires that the correlation between the two sources is 
known in the encoding process, a requisite that in many applications (e.g., when the nodes are randomly placed in 
an environment) can be hardly achieved. In an attempt to overcome this impairment, several papers have proposed 
ISCD schemes where the correlated sources are channel encoded at a reduced rate (with respect to the unconelated 
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case). The reduced reliability due to channel coding rate reduction can be compensated by exploiting correlation 
among different information sources at the channel decoder ifTTl . lITSl . |[T9l . ll20l . lIlTI . In particular, exploiting 
correlation by means of API has been shown to achieve very good performance. 

Although the great attention that has been given to these topics in the recent literature, the problem of designing 
good codes in presence of API has not been addressed so far. This is because it is generally assumed that good 
codes in the classical case (no API) are still good in presence of API. In an attempt to fill this lack, in this paper 
we derive some useful bounds for the bit error probability which establish that the performance depends not only 
on codewords' weights, as in traditional decoding, but also on information data weights. The proposed analysis 
allows to give an insight into the design of good codes, i.e., channel codes which permit to take the best advantage 
from exploiting API at the decoder. Furthermore, we consider the transmission of correlated binary sources from 
independent nodes and we propose a very simple JSCD scheme, where each decoder works by exploiting API 
given by the other decoder. 

This paper is structured as follows. In Section II, we derive the pairwise error probability in presence of API at 
the decoder. In Section III we validate the analysis in the uncoded case. In Section IV we provide an analytical 
study for evaluating performance in three different coded scenarios: (/) convolutional codes, (//) random codes with 
infinite length, and (///) turbo codes. Eventually, in Section IV we propose a JSCD scheme for decoding correlated 
binary sources from independent nodes. Finally, concluding remarks are given in Section IV. 



II. Pairwise error probability evaluation 

We consider an i.i.d binary source signal x of length k which is channel encoded with rate r = k/n and denote 
by c the binary coded signal of length n. We assume that a side-information f / = 0/1 about the message x is 
available at the decoder and we denote to as p the side-information reliability, i.e., p = Pr{xi =x,). Let introduce 
the a-priori log-likelihood terms L{xi) = In p^l^'Zij Qn represents the natural logarithm). Given these notations, 

it is easy to derive L{xi) = L{x) x (— 2jc/+ 1), where L{x) = In (^t^) ■ Of course, in order to fruitfully exploit the 
side information, the channel decoder must generate an estimate of the reliability p. This can be easily obtained 
by evaluating the number of zeros of the XOR between the received sequences. In the following, we assume that 
an estimation p is available at the decoder. Accordingly, we introduce L{x) = In {j^rp^- 

Let us denote by y(x) the transmitted signal and assume a binary antipodal modulation scheme, so that y(x) = 
— 2c(x) + 1. Eventually, assuming an AWGN channel model, we can express the received signal z as: 



z = X y(x) + T] (1) 

where T],- are Gaussian random noise terms with zero mean and variance Nq and is the energy per bit. 
Denoting by x the side information at the decoder, the MAP decoding rule can be expressed as: 

x = argmax Pr{x|p,x,z} (^2) 

By using the Bayes' rule and neglecting any constant term (i.e., the terms which do not depend on x), it is now 
straightforward to get from (|2]) the equivalent decoding rule: 

x = argmax Pr {z|x}/'r{x|p,x} 
Using the AWGN assumption and substituting for z the expression given in ([T|) it is easy to derive: 



: arg max 

X 





VMb L Ziyi+No X In (Pr{x|p,x}) 

1=0 



(4) 



Let us now denote by x, the transmitted information signal, and by Xg 7^ x, the estimated sequence. Moreover, let 
denote by 7^ the corresponding codewords. The pairwise error probability conditioned to x can be defined 
as the probability that the metric (01) evaluated for y = y^ and x = Xg is higher than that evaluated for y = yt and 
X = X(. Such a probability can be expressed as: 

Pe (x„x,|x) = Pr I V2^"l iyi,e-y.,,) -Noxln { p,\^%% ) >o| (5) 
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Substituting for z in ([5]) the expression given in ([T|), it is straightforward to obtain: 

P.(xr,x.|x) =0.5.r/c 1 /„(^^Wp|} 



(6) 



where 'Yb = d = D{ct,Ce) is the Hamming distance between Ct and Ce and erfc is the complementary error 
function. 

To elaborate, we get from the hypothesis that x is an i.i.d. sequence: 



Pr{x,\p,\} ^ Pr{xi,\p,Xi} 
M^elP^^ ~ Pr{xu\p,Xi\ 



(7) 



Let us introduce the sequences Ejj = Xij^Xi and £, e = Xi^g^xi, where is the bit- wise XOR operator. By 
exploiting the API x, and its estimated reliability p, the i-th term in <^ can be further elaborated as: 



Pr{xi,,\p,Xi} _ p%ty^{l-pfi,t 
PrlxulPA} p'''> X (l-pf''' 



1 if Xjj — Xjg 

if Xij ^ Xi^e and = 
^ if Xjj 7^ x,> and = 1 



(8) 



where e,^/ and £, e are the NOT version of Su and £,>, respectively. Hence, denoting by U (xr,Xg) the set of indexes 
such as Xi^t T^Xi^e, i-e-> xu y^Xi,e y i (x,,Xe), we can write: 

Pr{xij\p,Xi} 



Pr{x,\p,x} 



Pr{Xe\p,x} n PrIxu\p,Xi} 



(9) 



For the sake of notation clarity, we assume without loss of generality that U (x^ ,Xe) is the set {0, 1,... ,w —I}, w 
being the cardinality of U {xt,Xe), i.e., w = D(x,,Xg) is the Hamming distance between X; and Xg. Hence, we can 
write from ^ and Q: 



Pr{x,\p,x} 
Pr{Xe\p.x} 



P 

1-P 



W— 1 

/=0 X 



w— 1 



w—\ 



(10) 



Denoting for the sake of simplicity £, , = £,, remembering that L{x) = In yr^j > introducing the term w = 
it is now straightforward to rewrite (O as: 



L ■ 

;=0 



P,{xr,x,\x)=0.5erfc{\lrdrh{l + ^^^^^^^^^ 



(11) 



It can be observed from (fTTl) that, if we condition to vv, the pairwise error probability depends on d and w rather 
than on the whole transmitted and estimated sequences x, and Xg. It is then possible to write: 



Pg {d,w\^) = O.Serfc { ^/ rdy, ( 1 + ^^^^^^ 



(12) 



Note that, according to the correlation model, £, are i.i.d binary random term with Pr{ei = 0} = p and Pr{ei = 1} = 
1 — p. Hence, w is binomially distributed with parameters w and 1 — p, and the pairwise error probability can be 
eventually derived as: 



Pe{d,w)= 0.5 L erfci JrdYhh+^ 



w=Q 



1 L{x){w-2w) 



4rYb 



>"'-*x(l-p)'^ 



(13) 



The above expression is quite messy to manipulate. A significant simplification occurs if we consider the following 
bound: ^ 

' (14) 



, l L(x)iw-2w) 



> 1 1 2 L(x){w-2w) 
- d 



4rYh 



which is a tight lower bound for rdji, » \L{x) (w — 2w) \ , i.e., when the error probability is mainly determined 
by the codewords' distance rather than by the beneficial effect of API. In this case, we get: 



Pe{d,w) < 0.5 I erfc { ^rdyb + 

H'=0 



L(x)(w—2w) 



(;!;)p--x(i-p) 



(15) 



3 



To get the desirable simplification, consider now the Chemoff-Rubin bound for the erfc function, i.e.: 



Accordingly, we can write: 



which yields: 



Pe{d,w)<e-'''^^he 



erfc{x)<2e-' 



L(x)w W =, , , _ ... 
w=0 



L(.v) 



Since = ./ j2-^^ if we introduce the term: 



Pe{d,w) <e-'''TbA'' 



it is straightforward to get from (1181) : 



(16) 

(17) 
(18) 

(19) 
(20) 



The above expressions allows to separate the influence of signal to noise ratio and codewords distance d (first part) 
from the effect of API (second part). A more precise measure of the pairwise error probabiUty can be derived by 
considering the exact evaluation of the first term in ( |20l ) instead of its exponential bound, i.e.: 



(21) 



Note that ((21]) gives an exact calculation of the pairwise error probability for p = p = 0.5, i.e., in absence of API. 
Even if (|2T]) is not a strict bound for Pe {d,w), we will prove by simulations that it gives a quite close upper bound 
in most of the situations. 

Equations (l20l) and (|2T]) give rise to interesting considerations about the properties of good channel codes in presence 
of API. As in traditional codes' design, a good code must be characterized by a high minimum Hamming weight 
d. Moreover, in order to fully exploit the benefits of API, the code structure should allow to associate information 
sequences with high Hamming weights w to codewords with low Hamming weights d. This result can be easily 
understood if we rewrite dlOl) as: 



1 ^ 1 



(22) 



and if we observe that for reasonable p estimates, i.e., p = p, we get A < 1. Hence, denoting by a = ln(\) a 
rule of the thumb for designing good codes is that of maximizing the minimum d + wa (with a > 0). Of course, 
a rigorous analysis should consider the trade-off between diminishing the pairwise error probability from one side 
and increasing the number of bits in errors w from the other side. 



III. Uncoded Communications 

In the uncoded case r = k = n = I, d = w = I and the pairwise error probability is equivalent to the bit error 
probability, which can be derived according to (fT3] ) as: 



Pe = O.Serfc 7, (^1 + j j p + O.Serfc 

The approximation (|2T]) can be written in this case as: 

Pe,b {d,w) ~ O.Serfc I 




^) Ix(l-P) 



(23) 



(24) 

A comparison between the exact calculation in (1231 ) and the approximation in ((24)) is given in Fig. [T] In the y-axis 
we report the 7, required to achieve a target bit error probability, say it P^,,-. In the x-axis we report p. Four 
different p values have been considered, namely p = 0.5 in Fig. [J (a), p = 0.7 in Fig. [I](b), p = 0.9 in Fig. [I](c) 
and p = 0.95 in Fig. [T] (d). We note that approximation ((24|) is almost exact for p < 0.7. Moreover, it is a very 
close upper bound for p > 0.7 and for Pe^r = 0.001, i.e., for high signal to noise ratios. As expected, ((24l ) gives 
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a worse approximation for p = 0.9, p = 0.95 and for < 4, where the bound (fT4l ) is less tight. However, also 
in these cases, (l24l ) gives a quite close upper bound for the bit error probability evaluation. Hence, the proposed 
approximation allows to give a very good measure of the performance gain which can be obtained by exploiting 
API at the receiver, even in presence of imperfect estimation. Note that the system performance is quite robust to 
imperfect reliability estimation, at least for p < 0.95. As an example, for p = 0.9 and Pg r = 0.001, an estimation of 
p = 0.8 reduces the performance by only 0.1 dB with respect to perfect estimation (p = 0.9), while an estimation 
of p = 0.95 reduces the performance by less than 0.08 dB. To sum up, results in Fig. [T] show that API allows to 
achieve reasonable performance gains at low yt, with respect to the p = 0.5 case. This is true even in presence of 
not very accurate estimation of the side information reliability p. 



IV. Coded Communication Schemes 

A. Convolutional codes 



Convolutional coding schemes | |22| . 11231 allow an easy coding implementation with very low power and memory 
requirements and, hence, they seem to be particularly suitable for utilization in WSNs |[24ll . Moreover, as stated 
in the Introduction, correlation among sources may be directly converted to API at the receiver. Hence, optimum 
decoding schemes can be easily derived by including the a-priori probabilities in the branch metrics of the Viterbi 
algorithm according to equation (01). 

As in traditional convolutional coding (i.e., without API), it is possible to derive an upper bound of the bit error 
probability as the weighted sum of the pairwise error probabilities relative to all paths which diverge from the 
zero state and marge again after a certain number of transitions f22]. This is possible because of the linearity of the 
code and because the pairwise error probability ( fT3l) depends only on the weights d and w, and not on the actual 
transmitted sequence. 

In particular, it is possible to evaluate the input-output transfer function T{W,D) by means of the state transition 
relations over the modified state diagram [i22il . The generic form of T{W,D) is: 

r(W,D)= I/3v.,rfW-D'' (25) 

w.d 

where ^^.d denotes the number of paths that start from the zero state and reemerge with the zero state and that are 
associated with an input sequence of weight w, and an output sequence of weight d. Accordingly, we can get an 
upper bound of the bit error probability as: 

Pe,b < L Pw.d xwxPg{d,w) (26) 

w,d 

where Pe{d,w) is the pairwise error probability. Let now denote by Pe.e the exact pairwise error probability derived 
in ( fT3] ) and by Pg.a the approximation (|2TI ). Accordingly, we get the following bound for the bit error probability: 

Pe,l= L Pw,d X W X Pe^e {d,w) (27) 
w,d 

A second bound can be obtained by considering the loose upper bound (l20l ): 

Pe,2 = L j8„w X w X e-'-^M"' (28) 

w.d 

From ( |25l) and (|28] l it is straightforward to obtain: 

p -A.. dT(W.D) I .29) 



Since Pgj is a monotone decreasing function of jb, it is straightforward to carry out numerical inversion of (|29|) 
with respect to 7^. Such an inversion allows to get an estimation of the threshold signal-to-noise ratio 7^(P(, ,.,A) 
corresponding to a given ^(,,2 = Pe,r- Note that yh{Pe,ri 1) corresponds to the threshold 7, when no API is present at 
the receiver. Accordingly, the signal-to-noise-ratio gain due to API can be derived as: 

^=1^7) (30) 

'The weights are the information error weights 
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In order to assess the validity of the previous analysis, we have carried out computer simulations for both recursive 
and non-recursive convolutional codes. In both cases, we have considered a rate r = 0.5 and a constraint length K = 4. 
Hence, the codes can be univocally characterized by the generator polynomials G^^^ (D) = gf^D^ +g2^D^ +g[^^D + 

gll\ G^'^\D)=gf^ xD^+gf^D'^+gf^D+gf^ and by the feedback polynomial 7^ (D) = /z3 xD^+ZiiD^ + /iiD + /Jo- 
As for the non-recursive code we have considered the maximum dfree code which is optimum in the uncorrelated 
scenario ||23l, i.e., g(^'(D) =0^+0^ + 1, G^^\d) =D^+D^+D + l and, of course, H{D) = 1. Such a code is 
characterized by a transfer function: 

^(^' W^) = ""'T-iwVw"^' = D'W^ + D-^W + 2D^W3 + . . . (31) 

It is worth noting that the non-recursive code is characterized by a path with minimum distance d/ree = 6 and 
information weight w = 2. 

As for the recursive code, we consider the generator polynomials G^^\D) =D^+D + l, G'^'^\D) =D^ +D^+D + \ 
and H{D) = + 1. Such a code is characterized by a transfer function: 

= D^W^ + D^W^ + 2D^W^ + ... 

The recursive code is characterized by a path with minimum distance dfree = 6 and information weight w = 4. 
Given the above, for high signal to noise ratios the bit error probability can be approximated as Pg =A^erfc (\/37/,) 
for the non-recursive code and as Pg = lA'^erfc {yj'iyh) for the recursive code. Accordingly, we expect that the 
recursive code outperforms the non-recursive one for A < Under the hypothesis of perfect reUabiUty estimation, 
i.e., p = p, this means that the recursive code performs better for p > 0.85. 

Comparisons between the above codes are shown in Figs. |2][5] in the case of perfect reliability estimation and for 
different p values, namely p = 0.7 in Fig. [2l p = 0.8 in Fig. O p = 0.9 in Fig. |4] and p = 0.95 in Fig. [5] In all 
figures simulation results are shown together with the Pg.i upper bound derived in (|27]) . 

As one can observe, the analytical upper bound derived in Pg j is quite tight and, in particular, tends to perfectly 
match simulation results for high signal to noise ratios. Moreover, as expected, the recursive code clearly outperforms 
the non-recursive one for p > 0.9, while for p < 0.8 the non recursive code performs better. 
More extensive comparisons between simulations and theoretical analysis have been carried out to evaluate the 
signal to noise ratio gain AP which can be obtained by means of API at the receiver. Such results are shown in 
Fig. [6l where AP versus p for Pe,r = 0.0001 is shown. Simulation results are the straight lines while analytical 
results derived according to ( [30l ) are the dashed lines. Different p values have been considered, namely p = 0.8 
in Figs. [6] (a) and[6](b) and p =0.9 in Figs. |6](c) and[6](d). Eventually, results for the recursive code are shown 
in Figs. |6] (a) and [6] (c) and results for the non recursive code are shown in Figs. [6] (b) and |6] (d). We note that 
approximation (|29l ) allows to predict quite well the beneficial effect of a priori information even in the case of non 
perfect p estimation. It is worth noting that, as expected, recursive code takes grater advantage from exploiting a 
priori information than non recursive code. As an example, for p = 0.9 the maximum performance gain (i.e., the 
performance gain which is obtained for p = p) is 0.9 dB for the recursive code and 0.5 dB for the non recursive 
code. On the other hand, the recursive code is much more sensitive to estimation errors than the non recursive code 
(on account of the higher minimum w). 



B. Random Selection Of Codes 

In an attempt to derive a general framework for the evaluation of the impact of API in the performance of coded 
signals, we now consider random selection of codes and we evaluate a bound on the average bit error probability. In 
the proposed approach we extend the considerations made in |23|, Section 7-2, to the case of a priori information 
at the receiver. In particular, denoting by M = 2*, we consider the ensemble of (2")*^ distinct ways in which we 
can select M binary codewords from the available 2" words of length n. Each code selection leads to a different 
communication system which is characterized by its probability of error. As done in [23] we assume that the choice 
of M codewords is based on random selection. In particular, in Ii23il it is derived an upper bound on the expected 
pairwise error probabiUty for a given Hamming distance d as: 

P'e<^iOe-'-'^''' (33) 
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where the average is evaluated over the ensemble of (2")^ codes. Let now consider the upper bound derived in 
(I20I) for the pairwise error probability in presence of API. It is worth noting that in this case the pairwise error 
probability depends on d and w, whereas in absence of API it depends only on d. Moreover, since the code selection 
is random, d and w are binomial independent random discrete variables. Hence, averaging over the ensemble of 
(2")^ codes we get in this case: 

Pe<Hi L OOie-'^'fA- (34) 

J=Ow=0 

where A is defined in ( fT9l ). From the above, it is then straightforward to derive: 

^<(i±^)"x(i±A)^ (35) 

Eventually, since the average pairwise error probability is independent of d and w we can easily obtain an union 
bound on the average bit error probability by considering the sum of all the M —I possible error events, i.e.: 

n:^< (M-1) (i±^)" X m'<M(^y x {i^y (36) 



This result can be expressed in a more convenient form by introducing the terms Ri = log2 j ^^'^ V = 

log! ( ■ Accordingly, since M = 2* and r = k/n, (l36l ) becomes: 

^ 2nr-nRi-,ur] ^ 2-"[«l-'-(l-i7)] (37) 

We have thus obtained a similar expression for the average bit error probability as that in 1*231, with the introduction 
of the term r\ which takes into account the effect of API. Hence, introducing the cutoff rate Rq = we conclude 
that when r <Rq the average bit error probability P^j, — > as the code length « — > oo, i.e., there exist "good" codes 
that have a probability of error which goes to zero. 

In order to derive a measure of the performance gain which can be obtained by API, we introduce the term Ji, , as 
the minimum ji, which ensures the presence of a good codes for a given transmission rate r. It is straightforward 
to derive from the above: 

yhM) = -rl^{ 2^-riln)_, ) (38) 
The signal-to-noise-ratio gain due to API for a given r can be evaluated in this case as: 

^ ~ TVM ~ te(2l-'-(l-'))-l) ^^^^ 

It is now possible to get an insight into the performance of r = 0.5 convolutional codes presented in the previous 
Section where the AP gain has been evaluated for a target Pe,r = 0.0001. In particular, considering the case p =p = 
0.9 (i.e., T7 = 0.3219) and setting r = 0.5, we get from ( [39l ) AP = 2.1 dB, whereas the recursive convolutional code 
proposed in the previous Section yields AP = 0.9 dB and the non recursive convolutional codes yields AP = 0.5 
dB (see Fig. |6ll. 



C. Turbo codes 

In an attempt of reducing the gap between the theoretical AP derived in the previous Section and the actual 
AP which can be obtained by real codes, we analyze in this subsection the performance of parallel concatenated 
codes (turbo codes |[25l . |[26ll ) in presence of API at the decoder. As it is well known, the trick in turbo coding 
is to "statistically" break low weight codewords by means of random interleaving, so that the performance of the 
decoder in the region of not too much low BERs □ is mainly driven by high weights codewords (which occur with 
much higher probability than low weights codewords). On the other hand, since constituent codes are convolutional 
codes, high weights codewords are also characterized by high information weight (i.e., high w values). Hence, in 
this BER region, we expect that turbo codes allow to take the best advantage of exploiting API at the receiver. On 
the contrary, for random interleaving, the performance of Turbo codes at very low BERs is mainly dominated by 
low distance codewords |[27l . Such codewords are also characterized by small w values and hence we expect that 

^Not too much low BERs mean before approaching the well known error floor region of turbo codes. 
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in the error floor region the gain which can be obtained by exploiting API is small, i.e., similar to the gain that 
can be obtained by convolutional codes. 

To elaborate, let us consider a two-code turbo code with random interleaving and with identical constituent 
convolutional encoders. As it is discussed in [25], the weight 2 (i.e., w = 2) input data sequences which correspond 
to low weight codewords are the sequences which dominate the performance at low BER values. Let us denote by 
d2 the minimum codewords' weight which correspond to single error events of weight w = 2 in the trellis of the 
constituent codes. The minimum weight of the turbo code's codewords which corresponds to such w = 2 sequences 
is = 2d2 — 2. This distance is obtained when the same error event is presented at the input of the two encoders 
(it is two times d2 minus the information weight w, since the systematic bits are sent only once). The bit error 
probability of two-codes turbo codes in the error floor region, namely pj;^\ can then be approximated as: 

Pj)) ^ 2Ky0.5erfc (^n^ = Kierfc (^n^ (40) 

where Ki is the number of turbo coded sequences with information weight w = 2 and codeword's weight d2,t- For 
random interleaving it can be easily shown that Ki = 2/k [25 1. According to the analysis provided in the previous 
Sections, we then expect that the bit error probability in presence of API, namely Pff, is smaller than P^^f, i-c: 

^i/ =f^'-/c(v/n^) (41) 

where A is defined in ( fT9l ). 

As it is well known, performance of turbo codes can be improved by a more accurate design of the interleaver 
ETl . As an example, S-random interleavers [25] allow to avoid short cycle events, i.e., two bits which are close 
to each other both before and after interleaving. For comparison purposes, we then consider a specific interleaver 
derived by applying the S-random algorithm. 

Computer simulations of a two-code turbo code system with both random and S-random interleavers have then 
been carried out. The constituent codes are r = 1/2 recursive convolutional codes with constraint length K = A, 
G^'^\D) =D^+D^ + \, H{D) =D^+D + \, and G^'^\D) =H{D), (systematic code). The overaU rate of the turbo 
code is r = 1/3 which is increased to r = 1/2 via classical puncturing technique which enables to select the coded 
bits alternatively from the two encoders. The algorithm used by the two convolutional decoders at the receiver is 
based on the MAP BCJR scheme [28], which allows the inclusion of API in the form of LLRs of the input data. 
Fig. |7] show the BER versus ji, for the turbo codes {TC) introduced above. The frame size k of the information 
sequence (i.e., the interleaving size) is set to k= 1000 bits and the maximum number of iterations of turbo decoding 
is set to 10. Performance of random (l7](a)), and S-random (l2](b)) interleavers are shown for the case of no API, i.e., 
p = 0.5, and API with p = 0.9 and perfect estimation, i.e., p = p. Theoretical curves for the random interleaving 
evaluated according to (l40l) and (l4Ti ) are also shown. Note that for the considered code, K\ =2/1000 = 0.002. As far 
as d2,t is concerned, on account of puncturing we get d2,t = d2- The distance d2 can be easily computed by means 
of the modified state diagram ll22l . In particular, for the considered constituent codes we have d2 = ^, which yields 
d2.t = 8- Eventually, we also show the theoretical curves for the S-random case. In this case a performance analysis 
in the error floor region can be provided by following the WSE method proposed in [29|, where an union bound 
of the bit error probability is calculated as the partial sum of the dominant terms (corresponding to small code 
weights). Of course, we can also straightforwardly derive the bit error probability in presence of API by multiplying 
each term of the upper bound's partial sum by A"', w being the information weight of this term. Theoretical curves 
for the S-random case are denoted in Fig. |2]by Pg^\ for the p = 0.5 case, and pj^^ for the p = p = 0.9 case. 
Several comments can be drawn by the curves shown in Fig. |7] First of all note that, as expected, S-random 
interleaver allows to achieve performance better than random interleaver Moreover, for BER > 10^^ the considered 
turbo codes allow to exploit API much better than convolutional codes considered in the previous Section. As an 
example, if we consider Pe.r = 10^^ we observe that the performance gain due to API is higher than 1.6 dB for 
S-random interleaver and slightly lower than 1.5 dB for random interleaver El Similar gains are still achieved for 
Pe r = 10^^. This result is due to the fact that error events which mainly occur for such medium BER values are 
characterized by high w values. Instead, as expected, in the error floor region the curves for p = 0.5 and p = p = 0.9 
get closer since in this case the performance behavior is determined by low w error events. It is also worth noting 

■^Remember that recursive convolutional codes considered in this paper were able to achieve a performance gain of 0.9 dB 
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that the error floor fittings are very close to simulation results, thus confirming the validity of the proposed analysis. 
Results in Fig. |7] suggest that an accurate design of the interleaver in turbo codes may help the decoder to exploit 
better the API (if there is any). In particular, since the constituent codes of turbo codes are convolutional codes, 
the possibility of avoiding small w codewords is fully demanded to the possibility of the interleaver to break small 
weight input data sequences. Hence, even if the design of optimal interleavers in presence of API is out of the 
scope of this work, we can conclude that good interleaver for the classical case (no API) are good also for the case 
of API at the receiver. 

A question which arises from previous comments is wether turbo codes allow to approach the performance gain AP 
which has been derived in the previous Section for infinite length random codes. Of course the performance gain 
depends in general on the target BER P^, ,- that can be accepted. If we consider Pg^ = 10^^ we see from Fig. |7]that 
such a BER is quite close to the error floor region. To increase the AP for such a BER is then necessary to lower 
the error floor region, i.e., to decrease the probability of the occurrence of low w error events. As it is well known 
from the literature ll26l this can be easily obtained by increasing the frame size k. Hence we have run computer 
simulations for different k and for the S -random interleaver Results are summarized in Fig. |8] where AP versus p 
for Pe^r = 10"^ is shown for p = 0.7 (Fig. [8] (a)), p = 0.9 (Fig. [8] (b)) and for different k values, namely k = 100, 
k = 1000, and k = 100000. For comparison purposes, we also show AP of random codes (RC) with k = oo obtained 
through equation ( [39l ). Note that as k increases up to 100000, the performance gain due to API of TCs approach 
the theoretical gain of infinite length RCs. Of course this is true for P^ r = 10^^ while, for the considerations drawn 
before, it could not be true anymore for a lower BER target. It is also worth noting that the theoretical analysis for 
RCs gives an accurate bound of the allowable gains that can be obtained by exploiting API at the receiver even in 
presence of estimation errors. 



V. Case study: transmission of correlated signals observed at different nodes 

As discussed in the Introduction, the transmission of correlated signals observed at different nodes to one or 
more collectors has become a topical problem in the recent years, mainly because of the quick diffusion of Wireless 
Sensor Networks (WSNs). We consider in this Section a simple scenario where two independent nodes have to 
transmit correlated sensed data to a collector node. Such data, referred to as xi and yi, are taken to be i.i.d. correlated 
binary randon variables with Pr{xi = 1/0} = Pr{yi = 1/0} = 0.5 and correlation p = Pr{xi = yi} > 0.5. We consider 
a very simple Joint Source Channel Decoding (JSCD) technique where no source encoding is performed (i.e., no 
compression) but the two transmitters send their data over independent AWGN channels using the r = 1/2 punctured 
turbo code described in the previous Section. The independence of the noise terms in different links is due to the 
fact that the nodes are assumed to transmit over oithogonal multiple access channels (e.g., using frequency division 
multiple access). At the receiver two independent decoders performs an iterative decoding scheme where, at iteration 
m, the first decoder gives an estimation x- of Xi and the second decoder gives an estimation y - of y,-. To achieve 
this goal, the first/second decoder observes the signal coming from the first/second channel and performs turbo 
decoding taking y} 'Ix] as API. The correlation estimation p is evaluated at iteration m as: 

1- 1 A ®y (42) 
y — k 

Note that at first iteration (m = 0) neither the correlation nor the API are available at the two decoders and hence the 
first decoding step is performed by setting p^"' = 0.5. In this way the decoder does not need any knowledge about 
the correlation between the transmitter data. On the other hand, the theoretical analysis provided in the previous 
Sections show that the decoder performance is not very sensitive to estimation error (see Fig. [H). Hence, we expect 
that the decoder works well even in presence of imperfect correlation estimation and that it iteratively converges 
to achieve an almost perfect correlation estimation. 

We compare the proposed JSCD technique with the ideal separation-based strategy where the to-be-transmitted 
data are firstly compressed at the minimum achievable compression rate and then transmitted into the channel 
by means of turbo channel coding. Note that in this case the two transmitters must implement distributed source 
coding (DSC), and thus they must have a perfect correlation estimation (supposedly, correlation is still estimated 
at the receiver and then it is sent to the transmitters by means of a feedback channel). On the other hand, even 
in presence of perfect correlation estimation, the problem of designing good practical source codes for correlated 
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sources is still open. Hence, this second scheme can be considered as an ideal transmission scheme. In the DSC 
case, the two sources Xi and yi are independent (on account of compression) and, hence, decoding is performed 
without any API. To provide a fair comparison with the proposed JSCD technique we assume that in the separation 
case the rate of the channel encoder is lower, so that the global transmission rates is the same for the two cases. 
To elaborate, let assume a correlation p = 0.939 between the two sources. In this case the joint entropy of the 
two information signals is H{x,y) - H(x) + H{x\y) - I— p x log2{p) — (1 — p) x log2{l ^ p) - 133. This means 
that the two transmitters may achieve a compression rate of = 1.33/2 = 2/3 0. Hence, in order to achieve the 
same rate r= 1/2 as the JSCD case, in the separation case the channel coding rate may be set to 1/3. This can 
be achieved by using the unpunctured version of the turbo code described in the previous Section. Moreover, we 
consider the same signal-to-noise ratio SNR = Irjt, for JSCD and DSC, so that the two schemes are compared for 
the same overall transmitted rate and the same same total transmitted energy. Note that, since the channel rate in 
the DSC case is 3/2 times lower than in the JSCD case, the yt, value is 3/2 times higher (i.e, 1.76 dB higher). In 
other terms, we compare the rate r = 1/2 JSCD scheme with a given //, = 7 dB with the rate r = 1/3 DSC scheme 
with Yb = 7+ 1-76 dB. 

Fig. |9] show a BER comparisons between the JSCD and DSC scenarios described above. In particular, in Fig. |9] (a) 
we consider an AWGN channel model where the two channels are characterized by the same SNR. In Fig. |9] (b) we 
instead consider a block Rayleigh fading channel model H where SNR is exponentially distributed with the same 
average E{SNR) in the two channels. As far as the turbo code is of concern, the frame size k of the interleaver is 
set to ^ = 1000 bits and the maximum number of iterations is set to 10. 

Note that in the AWGN case, for a target Pg = 0.00001, the performance of the proposed JSDC scheme is only 0.2 
dB worse than the ideal DSC scheme. This assesses the validity of the proposed iterative JSCD scheme based on 
turbo coding. The most interesting and, dare we say, surprising results is derived in the Rayleigh case, where the 
JSDC decoding scheme clearly outperform DSC with a gain of more then 7 dB for = 0.001. The rationale for 
this result is that in presence of an unbalanced signal quality from the two transmitters (e.g., independent fading), 
leaving a correlation between the two information signals can be helpful since the better quality received signal 
can be used as side information for detecting the other signal. In other words, the proposed JSCD scheme allows 
to get a diversity gain K which is not obtainable by the DSC scheme. The diversity gain can be measured as the 
gradient of the BER curve, which yields K = I in the DSC case and K = 1.32 in the JSCD case. Such a diversity 
gain is due to the inherent correlation between information signals and, hence, can be exploited at the receiver 
without implementing any kind of cooperation between the transmitters. 

VI. Conclusions 

We have derived a novel analysis for evaluating decoding performance in presence of a-priori information with 
imperfect correlation estimation. According to this analysis, it is shown that the performance depends not only on 
the codewords' weight, as in traditional decoding, but also on the information data weight. We have then validated 
the proposed analysis in three different scenarios: convolutional codes, random codes and turbo codes. In particular, 
turbo codes have been shown to approach the performance of infinite length random codes. Moreover, we have 
proposed an effective joint source-channel decoding scheme in a wireless sensors network scenario where two 
nodes detect correlated sources and deliver them to a central collector. Experimental results show the the proposed 
scheme allows to approach the ideal Slepian-Wolf scheme in AWGN channel, and to clearly outperform it over 
fading channels on account of a diversity gain which can be achieved without implementing any kind of cooperation 
between the transmitters. 



We assume, as usually done for DSC, that the two transmitters use the same compression rate 
'The fading is assumed constant over the duration of a frame 
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Fig. 1. J), required to achieve Pe,r versus the estimated p (i.e., p is in the abscissa) in the uncoded case: comparisons between exact 
calculation (straight lines) and Pg^j (dashed Unes), for: (a) p = 0.5, (b) p = 0.7, (c) p = 0.9, (a) p = 0.95. 
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Fig. 2. Pe versus % comparisons between recursive and non recursive convolutional codes: simulation results are shown together with the 
Pe l upper bounds for p = 0.7 
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Fig. 3. Pe versus % comparisons between recursive and non recursive convolutional codes: 
Pe l upper bounds for p = 0.8 



simulation results are shown together with the 




Fig. 4. Pe versus % comparisons between recursive and non recursive convolutional codes: simulation results are shown together 
Pe l upper bounds for p = 0.9 
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Fig. 5. Pe versus % comparisons between recursive and non recursive convolutional codes: simulation results are shown together with the 
Pe l upper bounds for p = 0.95 
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Fig. 7. Pe versus Ji, for TC with L = 1000, random interleaving (a) and S-random interleaving (b): comparisons between no a-apriori 
(p = 0.5) and a-priori with p = p est. = 0.9. 



(a) p = 0.7 
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Fig. 8. AP versus the estimated p (i.e., p is in the abscissa) for P^.r = 0.00001: comparisons between random codes (RC) analysis with 
L = oo {39)1 and turbo codes (TC) with different L, for: (a) p = 0.7, (b) p = 0.9 
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Fig. 9. BER comparison between JSCD and DSC for : (a) AWGN (b) Rayleigh fading 



